Papers with task transfer
Universal Sentence Encoder for English (D18-2)
Copied to clipboard
Daniel Cer, Yinfei Yang, Sheng-yi Kong, Nan Hua, Nicole Limtiaco, Rhomni St. John, Noah Constant, Mario Guajardo-Cespedes, Steve Yuan, Chris Tar, Brian Strope, Ray Kurzweil
| Challenge: | TensorFlow Hub sentence embedding models have good task transfer performance . model variants allow for trade-offs between accuracy and compute resources . |
| Approach: | They propose easy-to-use TensorFlow Hub sentence embedding models with good task transfer performance. |
| Outcome: | The proposed models outperform models without transfer learning and those that use only word-level transfer on a number of NLP tasks. |
A Graph Interaction Framework on Relevance for Multimodal Named Entity Recognition with Multiple Images (2025.coling-main)
Copied to clipboard
| Challenge: | Existing methods to determine whether images are related to named entities are not effective in multi-image scenarios. |
| Approach: | They propose a graph interaction framework on relevance for Multimodal Named Entity Recognition with multiple images to integrate human abilities into the model. |
| Outcome: | The proposed framework achieves state-of-the-art on benchmark datasets and compares with CLIP and CLIP-based approaches. |
The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents (2020.acl-main)
Copied to clipboard
| Challenge: | a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy, ask questions, answer questions by utilizing knowledge resources, and perceive and converse about images. |
| Approach: | They propose a set of 12 tasks that measure if a conversational agent can communicate engagingly with personality and empathy . they use large dialogue datasets to multi-task and obtain state-of-the-art results . |
| Outcome: | The proposed model improves over a BERT pre-trained model on large dialogue datasets and provides state-of-the-art results on many of the tasks. |
Boosting Natural Language Generation from Instructions with Meta-Learning (2022.emnlp-main)
Copied to clipboard
| Challenge: | Recent work shows that language models trained with multi-task instructional learning (MTIL) can solve diverse NLP tasks in zero-shot settings with improved performance compared to prompt tuning. |
| Approach: | They propose to adapt meta-learning to MTIL in three directions: 1) Model Agnostic Meta Learning (MAML), 2) Hyper-Network adaptation to generate task specific parameters conditioned on instructions. |
| Outcome: | The proposed approaches improve over strong baselines in zero-shot settings and are most impactful when the test tasks are strictly zero- shot and are "hard" |
Multi-Agent Language Learning: Symbolic Mapping (2023.findings-acl)
Copied to clipboard
| Challenge: | Recent work has focused on the emergence of language in cooperative tasks where neural network agents learn a communication protocol from scratch to solve problems together. |
| Approach: | They propose a task transfer method and symbolic mapping architecture to help agents learn a compositional and symmetric language in dialog games. |
| Outcome: | The proposed method can help agents learn a compositional and symmetric language in complex settings like dialog games and the proposed architecture promotes vocabulary expansion. |
PICLe: Pseudo-annotations for In-Context Learning in Low-Resource Named Entity Detection (2025.naacl-long)
Copied to clipboard
| Challenge: | In-context learning is sensitive to the choice of demonstrations and can be used for tasks with few examples. |
| Approach: | They propose a framework for in-context learning with noisy, pseudo-annotated demonstrations . they annotate large quantities of demonstrations in a zero-shot first pass . |
| Outcome: | The proposed framework outperforms ICL on biomedical NED datasets with zero human-annotation. |
ScaLearn: Simple and Highly Parameter-Efficient Task Transfer by Learning to Scale (2024.findings-acl)
Copied to clipboard
| Challenge: | Multi-task learning (MTL) has shown significant practical benefits when using language models . current two stage MTL introduces a substantial number of additional parameters . |
| Approach: | They propose a multi-task learning method that leverages existing knowledge for a target task. |
| Outcome: | The proposed method outperforms baselines on three benchmarks and two encoder LMs with a small number of transfer parameters. |
Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark (2023.emnlp-main)
Copied to clipboard
| Challenge: | Existing benchmarks of social language are lacking for large language models. |
| Approach: | They propose a new benchmark that measures how well large language models understand social language by grouping 58 tasks into five categories: humor & sarcasm, offensiveness, sentiment & emotion, and trustworthiness. |
| Outcome: | The proposed model performs well at 58 tasks that are divided into five categories: humor & sarcasm, offensiveness, sentiment & emotion, and trustworthiness. |
Raise a Child in Large Language Model: Towards Effective and Generalizable Fine-tuning (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent pretrained language models extend from millions to billions of parameters. |
| Approach: | They propose a technique which forwards on a whole network while backwarding on resetting the gradients of the non-child network during the backward process. |
| Outcome: | The proposed technique outperforms the vanilla fine-tuning technique on various downstream tasks and can achieve better generalization performance by large margins. |
FETA: A Benchmark for Few-Sample Task Transfer in Open-Domain Dialogue (2022.emnlp-main)
Copied to clipboard
Alon Albalak, Yi-Lin Tuan, Pegah Jandaghi, Connor Pryor, Luke Yoffe, Deepak Ramachandran, Lise Getoor, Jay Pujara, William Yang Wang
| Challenge: | Prior studies of task transfer in dialogue consider only 2-4 tasks, focus on multitasks. |
| Approach: | They propose a benchmark for FEw-sample TAsk transfer in open-domain dialogue. |
| Outcome: | The proposed benchmark analyzes the transferability between 132 source-target task pairs and provides a baseline for future work. |